Older versions of the ROUGEeval summarization evaluation system were easier to fool
نویسنده
چکیده
We show some limitations of the ROUGE evaluation method for automatic summarization. We present a method for automatic summarization based on a Markov model of the source text. By a simple greedy word selection strategy, summaries with high ROUGE-scores are generated. These summaries would however not be considered good by human readers. The method can be adapted to trick different settings of the ROUGEeval package.
منابع مشابه
A survey on Automatic Text Summarization
Text summarization endeavors to produce a summary version of a text, while maintaining the original ideas. The textual content on the web, in particular, is growing at an exponential rate. The ability to decipher through such massive amount of data, in order to extract the useful information, is a major undertaking and requires an automatic mechanism to aid with the extant repository of informa...
متن کاملAchieving Integrated Care for Older People: What Kind of Ship?; Comment on “Achieving Integrated Care for Older People: Shuffling the Deckchairs or Making the System Watertight for the Future?”
This paper considers an implication of the idea that proposals for integrated care for older people should start from a focus on the patient, consider co-production solutions to the problems of care fragmentation, and be at a system-wide, cross-organisational level. It follows that the analysis, design and therefore evaluation of integrated care projects should be based upon the journeys which ...
متن کاملمقایسه روش‌های مختلف یادگیری ماشین در خلاصه‌سازی استخراجی گفتار به گفتار فارسی بدون استفاده از رونوشت
In this paper, extractive speech summarization using different machine learning algorithms was investigated. The task of Speech summarization deals with extracting important and salient segments from speech in order to access, search, extract and browse speech files easier and in a less costly manner. In this paper, a new method for speech summarization without using automatic speech recognitio...
متن کاملTask-Based Evaluation of Anaphora Resolution: The Case of Summarization
One of the types of semantic interpretation processes that may help ‘crossing the barriers’ in text summarization is anaphora resolution. In this paper, we show that summarization is a good task for evaluating the performance of an anaphoric resolver, in the sense that it encourages developing anaphoric resolvers that build good-quality discourse models, and the performance of the anaphoric res...
متن کاملText Summarization Using Cuckoo Search Optimization Algorithm
Today, with rapid growth of the World Wide Web and creation of Internet sites and online text resources, text summarization issue is highly attended by various researchers. Extractive-based text summarization is an important summarization method which is included of selecting the top representative sentences from the input document. When, we are facing into large data volume documents, the extr...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- Inf. Process. Manage.
دوره 43 شماره
صفحات -
تاریخ انتشار 2007